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A  VALID,  CULTURE-FAIR  TEST  OF  INTELLIGENCE 


EXECUTIVE  SUMMARY 


The  question  the  present  research  addressed  was  whether  a  racially  unbiased  test  of  the 
ability  to  process  information  would  predict  how  well  young  adults  succeed  in  college  classes. 
The  technical  barrier  overcome  was  that  current  theories  of  intelligence  are  based  on  an 
assumption  that  all  those  taking  IQ  tests  have  had  equal  opportunity  for  exposure  to  the 
information  being  tested.  Thus,  past  efforts  to  develop  an  intelligence  test  that  is  culture-fair  have 
not  been  successful.  The  significance  of  the  research  is  that  it  provides  further  evidence  to 
evaluate  a  theory  that  defines  intelligence  as  information  processing  ability  (Fagan,  1992,  2000). 
Current  research  on  a  theory  of  intelligence  as  information  processing  finds  racial  differences  in 
IQ  to  be  due  to  cultural  factors.  A  test  of  information  processing  is  the  first  valid,  culture-fair  test 
of  intelligence. 

Research  Requirement: 

f 

The  present  research  is  based  on  a  theory  (Fagan,  1 992,  2000)  that  assumes  that  we  act  on 
the  basis  of  what  we  know.  What  we  know  is  a  result  of  our  processing  of  the  information 
provided  to  us  by  our  culture.  Based  on  these  assumptions,  the  IQ  score  is  seen  as  a  measure  of 
knowledge.  Specifically,  how  much  you  know  (your  IQ)  depends  on  how  well  you  process 
information  (your  intelligence)  and  also  on  the  information  given  to  you  by  your  culture  to 
process.  The  theory  has  been  used  to  explain  the  source  of  IQ  differences  between  Americans  of 
different  races.  Specifically,  past  research  has  found  (Fagan  &  Holland,  2002,  2007)  that 
American  Whites  and  African  Americans  who  differ  in  IQ  do  not  differ  in  their  ability  to  process 
information  when  that  information  has  been  equally  accessible  to  both  groups. 

Procedure: 

College  students  were  given  multiple  choice  tests  of  their  ability  to  acquire  new 
information  concerning  the  meanings  of  previously  unknown  words,  sayings,  similarities,  and 
analogies.  They  also  were  tested  for  their  knowledge  of  vocabulary,  opposites,  and  analogies 
with  a  brief  version  of  the  Scholastic  Assessment  Test-Verbal  (SAT-V)  constructed  for  the 
present  contract.  The  brief  SAT-V  was  constructed  because  some  of  the  participants  were 
community  college  students  who  are  not  required  to  take  the  SAT  for  admission  to  college  and  a 
standard  measure  of  academic  aptitude  that  would  be  common  to  both  community  college 
students  and  university  students  was  needed.  Numerical  grades  in  courses  were  obtained  for  the 
majority  of  participants.  Associations  among  performance  on  the  culture-fair  tests  of  new 
learning,  academic  aptitude  (the  brief  version  of  the  Scholastic  Assessment  Test  scores),  and 
specific  achievement  (objective  test  scores  in  college  courses)  were  analyzed. 

Findings: 

The  contract  was  undertaken  to  discover  whether  a  measure  developed  to  assess  the 
ability  to  process  new  information  is  both  valid  and  culture-fair.  The  findings  reveal  that  tests  of 
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new  learning  are  culture- fair,  reliable,  and  predictive  of  both  academic  achievement  (numerical 
grades  in  class)  and  of  a  brief  version  of  a  standard  test  of  scholastic  aptitude  (the  Scholastic 
Assessment  Test-Verbal).  Further,  the  results  demonstrate  that  tests  of  new  learning  and  tests  of 
existing  knowledge  (such  as  the  brief  SAT)  each  contribute  independent  variance  to  the 
prediction  of  class  grades.  A  final  notable  finding  is  that  the  brief,  24  item  version  of  the  SAT-V 
that  was  created  for  the  present  investigation  turns  out  to  be  as  predictive  of  college  grades  as  the 
standard  SAT-V.  In  summary,  the  present  research  indicates  that  there  are  at  least  two  factors 
that  determine  success  in  a  complex  learning  environment  such  as  higher  education:  one  is 
information  processing  (new  learning)  ability  and  the  other  is  the  extent  of  knowledge  one  has 
acquired  to  that  point. 

Fagan  (2000)  assumes  that  the  failure  to  develop  tests  of  intelligence  that  can  be  fairly 
applied  across  racial  groups  stems  from  a  theoretical  bias  to  equate  the  IQ  score  with  intelligence 
rather  than  with  knowledge.  If  intelligence  is  defined  as  information  processing  and  the  IQ  score 
as  knowledge,  the  possibility  of  culture-fair  tests  of  intelligence  based  on  estimates  of 
information  processing  arises.  The  chief  theoretical  implication  of  the  results  of  the  present 
research  is  that  the  assumption  of  equal  opportunity  for  exposure  to  information  made  by 
theories  of  intelligence  is  false.  When  such  an  assumption  is  shown  to  be  false,  it  becomes 
possible  to  develop  culture-fair  tests  of  intelligence  that  allow  basic  intellectual  abilities  to  be 
recognized. 

The  general  purpose  of  the  present  contract  is  to  add  fundamental  knowledge  to 
behavioral  science...  and  to  discover  general  principles  ...  [through  an]  integrated 
programmatic  effort  to  develop  and  to  test  theory  (as  called  for  by  BRU  in  BRO-BAA,  page  1). 
The  specific  goal  is  to  discover  the  relationships  among  information  processing  ability  and 
knowledge  that  lead  to  achievement  and  productivity  in  complex  situations.  The  goal  is  in 
keeping  with  the  Army’s  goal  to  improve  its  ability  to  select,  classify,  train,  and/or  develop 
Soldiers  and  leaders  who  ...are  adaptable... [and  who]...  can  function  effectively  in 
...information  rich  ...environments  (BRO-BAA,  page  1,  II,  1,  a,  b).  One  of  the  BRO-BAA  Basic 
Research  Areas  of  Interest  is  leadership  skills.  The  research  will  aid  in  the  goal  of  identifying 
leadership  skills  in  adaptability  ...when  faced  with  novel  situations  (BRO-BAA,  page  4,  III,  C, 

1).  The  research  has  resulted  in  a  reliable,  valid,  and  culture-fair  test  of  intelligence  based  on  the 
ability  to  process  new  information.  The  research  aids  in  further  demonstrating  that  intelligence  is 
a  function  of  an  aptitude  [learning  ability  in  novel  situations]  that  cuts  across  domains  and  how 
to  develop  a  method  for  measuring  this  aptitude  [and  to  further]  validate  [this]  cognitive  ability 
against  relevant  criteria  (i.e.,  academic  achievement),  (BRO-BAA,  page  5,  III,  D,  1). 

In  summary,  the  research  is  in  keeping  with  the  Army’s  goal  to  select  Soldiers  and 
leaders  who  can  function  effectively  in  information  rich  environments  (BRO-BAA,  page  1,  II,  1, 
a,  b)  and  to  identify  leadership  skills  as  to  adaptability  when  faced  with  novel  situations  (BRO- 
BAA,  page  4,  III,  C,  1).  The  proposed  research  further  aids  in  (1)  demonstrating  that  intelligence 
defined  as  learning  ability  in  novel  situations  is  an  aptitude  that  crosses  domains  and  (2) 
validating  this  cognitive  ability  against  relevant  criteria  (BRO-BAA,  page  5,  III,  D,  1). 
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Utilization  and  Dissemination  of  Findings: 

Two  of  the  guiding  missions  of  AR1,  both  historically  and  currently,  are  to  provide  new 
technology  to  meet  the  personnel  challenges  of  the  Army  in  recruiting,  selecting,  and  assigning 
quality  people  and  to  provide  scientific  insight  into  leader  development.  The  present  research 
provides  basic  scientific  data  that  may  be  applied  by  ARI  to  aid  in  Soldier  selection,  assignment, 
and  performance  and  in  leader  development.  The  results  of  the  present  contract  may  lead  to 
applied  research  that  would  be  meaningful  to  the  Army  in  the  development  of  new,  applied 
behavioral  technologies  to  identify  individual  differences  in  cognitive  ability  that  may  be  used  in 
the  selection  of  Soldiers  and  in  the  identification  of  mature,  effective  leaders.  In  addition,  the 
proposed  research  will  provide  data  to  aid  such  selection  and  identification  to  be  culture-fair  with 
regard  to  gender  and  race.  In  general,  tests  of  information  processing  ability  that  are  valid 
predictors  of  achievement  in  complex  situations  will  have  important  military  applications.  Such 
tests  can  aid  in  providing  a  valid  means  to  evaluate  volunteers  and  select  candidates  for  advanced 
education  or  training  in  complex  situations  of  the  sort  performed  by  Army  personnel.  Such  tests 
also  can  be  employed  in  further  selection  and  training  stages.  In  the  field,  such  skills  are 
necessary  to  make  quick  and  correct  decisions  based  on  rapidly  incoming  information.  Skill  in 
information  processing  is  likely  to  be  a  key  indicator  of  effective  future  Army  leaders.  Finally, 
using  tests  of  information  processing  known  to  be  culture-fair  may  provide  an  increase  in  the 
number  of  eligible  recruits  and  an  incentive  to  re-enlistment  on  the  part  of  minorities. 
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Introduction 


Current  tests  of  intelligence  are  culturally  biased.  The  purpose  of  the  present  research  was 
to  test  the  predictive  validity  of  a  culture-fair  test  of  intelligence  based  on  the  ability  to  process 
new  information.  The  present  research  evaluates  a  theory  (Fagan,  2000)  that  defines  intelligence 
as  information  processing  ability.  In  theory,  the  IQ  score  is  seen  as  a  measure  of  knowledge 
resulting  from  processing  ability  and  the  information  provided  by  the  culture  for  processing. 

Brief  tests  of  knowledge  based  on  word  meanings,  similarities,  sayings,  and  analogies  that  have 
recently  been  shown  to  be  racially  unbiased  were  administered  to  young  adults.  Associations 
between  performance  on  culture-fair  tests,  academic  aptitude  (Scholastic  Assessment  Test 
scores),  and  specific  achievement  (grades  in  college  courses)  were  analyzed.  Practically,  if 
culture-fair  tests  of  information  processing  are  valid  predictors  of  achievement,  such  tests  can  aid 
in  providing  a  culturally  unbiased  means  to  select  candidates  for  advanced  education  or 
employment.  Differences  in  IQ  between  African  Americans  and  Whites  are  on  the  order  of  about 
1 5  points  (Jensen,  1985:  Fagan  &  Holland,  2002).  Are  racial  differences  in  IQ  due  to  differences 
in  innate  intellectual  ability  or  to  cultural  variations  in  exposure  to  particular  information?  There 
is  no  agreed  upon  answer  to  the  controversial  issue  of  the  source  of  racial  differences  in  IQ.  As 
Sternberg,  Grigorenko,  &  Kidd  (2005)  note,  we  first  need  to  know  what  intelligence  is  to 
understand  the  source  of  racial  differences  in  IQ.  Cooper  (2005)  agrees  that  new  theoretical 
approaches  to  the  question  of  the  sources  of  racial  equality  in  IQ  are  needed.  In  this  research,  we 
offer  a  theoretically  guided,  experimental  approach  to  the  question  of  the  basis  of  racial 
differences  in  IQ. 

Jensen  (1998)  assumes  that  the  opportunity  for  exposure  to  the  information  being  tested 
on  conventional  IQ  tests  has  been  the  same  for  all  races.  Given  that  assumption,  Jensen  attributes 
racial  differences  in  IQ  to  differences  in  basic  intellectual  ability.  In  contrast  to  Jensen,  Fagan 
assumes  (Fagan,  1984;  1991;  1992;  2000;  Fagan  &  Haiken-Vasen,  1997)  that  the  IQ  score  is  a 
measure  of  knowledge.  How  much  a  person  knows  (their  IQ)  depends  on  the  person’s 
information  processing  ability  (their  intelligence)  and  on  the  information  the  person  has  been 
given  to  process.  In  this  view,  if  group  differences  in  IQ  are  not  accompanied  by  group 
differences  in  information  processing  ability,  then  the  search  for  the  causes  of  the  IQ  differences 
should  be  directed  toward  differences  in  access  to  information. 

Fagan  and  Holland  (2002)  investigated  the  contributions  intellectual  ability  and  access  to 
information  make  to  racial  differences  in  IQ.  The  African  Americans  and  Whites  in  the  Fagan 
and  Holland  (2002)  studies  were  highly  representative  of  the  U.S.  population  with  regard  to  age 
and  education  (U.S.  Census  Bureau,  1999).  Moreover,  the  IQs  of  the  participants  in  the  Fagan 
and  Holland  (2002)  studies  were  representative  of  the  IQs  of  Whites  and  African  Americans  in 
the  U.S.  population.  These  young  adults  were  compared  for  their  knowledge  of  the  meanings  of 
words,  a  task  commonly  used  on  standard  IQ  tests  and,  most  importantly,  a  task  that  typically 
results  in  racial  group  differences  in  IQ.  Fagan  and  Holland  (2002)  ensured  that  the  African 
Americans  and  Whites  were  given  equal  opportunity  to  learn  the  meanings  of  novel  words  and 
conducted  tests  to  determine  how  much  knowledge  had  been  acquired.  For  example,  participants 
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might  be  exposed  to  a  sentence  such  as  “Tubby  had  a  big  fat  venter”  and  asked  to  indicate 
whether  a  “venter”  was  a  body  part  or  a  mental  state.  On  a  later  test  of  vocabulary  knowledge, 
the  word  "venter"  would  be  presented  with:  “a.  height,  b.  candle,  c.  badge,  d.  belly,  and  e. 
opening”  as  choices. 

If,  as  Jensen  suggests,  racial  differences  in  IQ  are  due  to  differences  in  intellectual  ability 
per  se,  then  knowledge  for  word  meanings  learned  under  exactly  the  same  conditions  should 
differ  between  African  Americans  and  Whites.  However,  if  differences  in  IQ  between  races  are 
due  to  unequal  opportunity  for  exposure  to  information,  rather  than  to  differences  in  intellectual 
ability,  no  differences  in  knowledge  should  be  obtained  between  African  Americans  and  Whites 
given  equal  opportunity  to  leam  new  information. 

Fagan  and  Holland  (2002)  controlled  for  the  possibility  that  the  particular  people  chosen 
to  represent  each  racial-ethnic  group  might,  by  chance,  simply  have  been  equal  in  vocabulary 
knowledge  initially.  Thus,  the  tests  of  a  person's  knowledge  of  the  newly  learned  words  were 
intermixed  with  similar  multiple  choice  tests  for  knowledge  of  the  meanings  of  different  words, 
words  for  which  no  special  training  in  the  experimental  situation  had  been  provided,  such  as 
those  typically  used  in  vocabulary  tests  of  IQ. 

In  accord  with  the  general  literature,  it  was  assumed  that  the  racial-ethnic  groups  would 
differ  in  knowledge  of  the  meanings  of  words  taken  from  standard  IQ  tests.  The  question  was 
whether  they  would  differ  in  their  knowledge  for  newly  learned  words.  The  Whites  were,  as 
expected,  superior  to  the  African  Americans  in  untrained  vocabulary  knowledge.  Conversely, 
when  equal  opportunity  for  exposure  to  the  meanings  of  words  was  experimentally  assured,  the 
Whites  and  the  African  Americans  were  equal  in  vocabulary  knowledge.  Thus,  if  members  of 
different  racial  groups  are  given  equal  opportunity  to  acquire  the  meanings  of  words,  their 
knowledge  of  the  meanings  of  these  words  does  not  differ.  Fagan  and  Holland  (2002)  concluded 
that  African  Americans  and  Whites  do  not  differ  in  the  ability  to  process  new  information  and 
that  the  search  for  racial  differences  in  knowledge  (IQ)  should  be  aimed  at  differences  in  the 
information  to  which  people  from  different  racial  groups  have  been  exposed. 

A  further  study  (Fagan  &  Holland,  2007)  explored  the  generality  of  the  Fagan  and 
Holland  (2002)  findings.  African  Americans  and  Whites  were  tested  for  their  knowledge  of 
sayings,  analogies,  or  similarities.  Material  was  presented  in  such  a  way  that  knowledge  of  the 
concepts  and  terms  employed  in  each  test  were  commonly  available  for  individuals  of  either 
race.  Participants  also  were  tested  for  their  understanding  of  sayings,  similarities,  and  analogies 
as  typically  given  in  assessments  of  IQ.  Knowledge  of  sayings,  similarities,  and  analogies  are 
commonly  used  measures  of  IQ  and  vary  with  race  (Jensen,  1980;  1981).  As  in  the  Fagan  and 
Holland  (2002)  study,  knowledge  such  as  that  tested  on  conventional  IQ  tests  varied  with  race, 
while  knowledge  based  on  information  made  generally  available  did  not  vary  with  race. 

In  brief,  the  data  of  Fagan  and  Holland  (2002,  2007)  support  the  view  that  cultural 
differences  in  the  provision  of  information  account  for  racial  differences  in  IQ.  Specifically,  the 
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results  indicate  that  IQ  differences  among  races  have  to  do  with  experience.  The  present  research 
focuses  on  the  implications  that  the  results  of  the  Fagan  and  Holland  studies  have  for  the 
development  of  a  valid,  culture-fair  test  of  intelligence. 

The  authors  of  standard  intelligence  tests  assume  that  a  person  who  knows  more  than 
another  person  about  particular  information  (such  knowledge  being  predictive  of  achievement  in 
that  culture)  is  the  more  intelligent  person.  As  Sternberg  (2000)  points  out,  the  processes  of 
intelligence  may  be  the  same  from  culture  to  culture,  but  a  person  is  called  more  or  less 
intelligent  based  on  socially  approved  standards  of  what  is  important  to  know.  Sternberg  goes  on 
to  note  that  confusing  intelligence  with  what  society  says  is  intelligent  may  cause  us  to  give  up 
on  people  who  have  basic  abilities  that  go  unrecognized.  Sternberg’s  observations  have  much 
merit.  The  chief  practical  implication  of  the  Fagan  and  Holland  studies  noted  above  is  that  it  may 
be  possible  to  develop  culture-fair  tests  of  intelligence  that  will  allow  basic  intellectual  abilities 
to  be  recognized.  Specifically,  the  inequality  of  educational  achievement  among  races  in  our 
country  has  highlighted  a  need  for  culture- fair  tests  of  intelligence.  African  Americans  do  not  do 
as  well  as  Whites  on  IQ  tests  and  other  tests  of  knowledge,  such  as  the  Scholastic  Assessment 
Test  and  the  Graduate  Record  Exam.  Basing  admission  to  higher  education  on  such  test  scores 
means  that  only  a  small  percentage  of  Blacks  are  eligible  for  admission  to  colleges  and 
universities.  A  culture-fair  test  of  intelligence  would  allow  basic  abilities  to  be  measured  and 
would  allow  those  with  appropriate  intellectual  skills  to  pursue  further  schooling. 

Jensen  (2000)  believes  that  it  may  not  be  possible  to  come  up  with  tasks  that  show  no 
differences  in  test  performance  between  African  Americans  and  Whites  and  yet  still  predict 
academic  performance.  Fagan  (2000)  does  not  agree  with  Jensen,  pointing  out  that  the  failure  to 
develop  tests  of  intelligence  that  can  be  fairly  applied  across  racial  groups  stems  from  a 
theoretical  bias  to  equate  the  IQ  score  with  intelligence  rather  than  with  knowledge.  If 
intelligence  is  defined  as  information  processing  and  the  IQ  score  as  knowledge,  the  possibility 
of  culture-fair  tests  of  intelligence  based  on  estimates  of  information  processing  arises.  Fagan  is 
not  alone  in  such  a  conjecture.  Williams  (2000,  p.17)  notes  that  “Fagan’s  ideas”  (Fagan  2000)  of 
measuring  thinking  or  information  processing  rather  than  accumulated  knowledge  are  “relevant 
to  the  debate  on  intelligence  testing  and  affirmative  action  because  ...  a  true  measure  of 
processing  efficiency  (if  it  could  be  devised)  would  be  fair  to  members  of  all  racial  and  ethnic 
groups.” 


In  the  present  research,  college  students  were  tested  for  their  ability  to  acquire  new 
information  concerning  the  meanings  of  previously  unknown  words,  sayings,  similarities,  and 
analogies.  They  also  were  tested  for  their  knowledge  of  vocabulary  in  general,  opposites,  and 
analogies  via  a  brief  version  of  the  verbal  section  of  the  Scholastic  Assessment  Test  (SAT-V) 
constructed  for  the  purposes  of  the  present  research.  For  a  small  number  of  the  participants, 
standard  SAT-V  scores  were  available.  Numerical  grades  in  courses  were  obtained  for  the 
majority  of  the  participants.  Associations  among  performance  on  the  culture-fair  tests  of  new 
learning,  academic  aptitude  (the  brief  SAT),  and  academic  achievement  (objective  test  scores  in 
college  courses)  were  analyzed.  The  goal  of  the  present  investigation  was  to  discover  if  a  culture- 
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fair  test  of  intelligence  based  on  new  learning  is  predictive  of  academic  achievement.  New 
learning  was  chosen  to  test  information  processing  ability  because  African  Americans  and 
Whites,  as  we  have  noted,  do  not  differ  on  such  ability.  Yet,  as  Fagan  and  Holland  found,  such 
new  learning  is  predictive  of  scores  on  standard  IQ  tests.  Practically,  measuring  new  learning  of 
initially  unknown  terms  allows  for  a  wealth  of  items  for  the  creation  of  alternate  forms  of 
culture- fair  tests  of  new  learning. 


Experiment  1 

The  purpose  of  an  initial  experiment  was  to  test  the  predictive  validity  of  a  culture-fair, 
racially  unbiased,  test  of  intelligence.  Specifically,  the  purpose  was  to  discover  if  a  culture-fair 
test  of  intelligence  based  on  learning  the  meanings  of  new  words,  sayings,  similarities,  and 
analogies  is  predictive  of  both  a  standard  assessment  of  scholastic  aptitude  and  of  academic 
achievement. 

Participants 

A  sample  of  484  students  (266  females,  218  males,  436  Whites  or  Asians  and  48  Blacks 
or  Hispanics)  with  a  mean  age  of  20.15  years  (SD  4.1  years)  and  a  mean  education  of  14.1  years 
(SD  1 .4  years)  took  part  in  Experiment  1 .  A  total  of  386  of  the  students  were  enrolled  at  private 
universities  (337  at  one  and  49  at  another),  22  attended  a  local  college,  and  46  attended  a  two- 
year  community  college.  All  participants  were  registered  for  psychology  classes.  Specific 
demographic  characteristics  were  gathered  by  asking  students  to  complete  a  form  that  asked  for 
their  age,  gender,  and  years  of  education.  Racial  identity  was  voluntarily  provided  by  the 
students,  who  read  and  filled  out  a  form  stating:  “According  to  the  United  States  Public  Health 
Service,  ‘women  and  members  of  minority  groups  and  their  subpopulations  must  be  included  in 
all  National  Institutes  of  Health  supported  biomedical  and  behavioral  research,’  and  researchers 
are  asked  to  ‘describe  the  composition  of  the  proposed  population  in  terms  of  gender  and 
racial/ethnic  group.’  To  aid  us  in  following  the  US  Public  Health  Service  guidelines  for  research, 
please  check  the  appropriate  category.”  The  student  then  checked  one  of  five  categories  labeled 
“American  Indian  or  Alaskan  Native,”  “Asian  or  Pacific  Islander,”  “Black,  not  of  Hispanic 
origin,”  “Hispanic,”  or  “White,  not  of  Hispanic  origin.” 

Materials 

Materials  included  a  culture-fair  test  of  the  students’  ability  to  acquire  new  information 
concerning  the  meanings  of  previously  unknown  words,  sayings,  similarities,  and  analogies. 
Examples  will  be  noted  below.  In  addition  to  the  culture-fair  tests  of  new  learning,  all 
participants  were  given  a  brief  SAT-type  test  based  on  questions  available  from  a  variety  of 
books  containing  practice  questions  for  the  SAT.  (Again,  examples  will  be  given  below).  The 
brief  SAT  was  given  to  ensure  that  all  participants  would  be  given  the  same  test,  since  standard 
SAT  scores  from  the  community  college  students  could  not  be  obtained  (they  are  not  required  to 
take  such  tests  for  admission).  In  addition,  students  from  other  countries  and  transfer  students  did 
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not  always  have  SAT  scores  on  record.  Standard  SAT-V  scores  were  available  for  291  of  the 
students  at  one  of  the  participating  universities.  Measures  of  specific  academic  achievement 
(exam  performance  on  objective  tests)  were  obtained  from  instructors  of  Psychology  courses 
who  were  asked  to  provide  the  students’  numerical  scores  on  the  tests  in  the  course.  Such  tests 
tended  to  be  objective  tests  (multiple  choice  and  true-false  questions)  based  on  lectures  and 
readings.  Grades  were  obtained  from  one  Psychology  class  per  participant,  typically  classes  in 
Introductory  Psychology,  and  were  usually  based  on  some  200-300  items  across  tests  during  a 
semester. 

Procedure 

Culture-fair  tests  of  new  knowledge  were  based  on  a  training  phase  and  a  testing  phase. 
All  training  and  testing  was  done  in  a  group  setting  during  set-aside  class  time. 

Training  for  the  learning  of  the  meanings  of  new  words,  consisted  of  a  form  that  said: 


"Now  you  are  going  to  see  how  some  unusual  words  are  used  in  sentences.  Read  each 
sentence  carefully.  Circle  a  or  b  to  answer  the  question  that  follows  each  sentence.  ” 

It  cost  1,500  bezants  to  buy  the  rug  in  Byzantium.  BEZANT  is:  a.  an  action  b.  an  object 


Training  would  then  continue  for  the  remaining  six  words  of  the  seven-item  set. 
Instructions  and  one  example  of  training  for  learning  the  meaning  of  new  sayings  follow: 


“On  the  following  pages  you  will  see  how  8  sayings  from  various  cultures  are  explained  in 
English.  Carefully  read  the  explanation  for  each  saying.  Then  circle  the  number  from  1  to  5 
that  indicates  how  well  you,  yourself,  now  understand  the  saying  from  the  explanation.  Please 
rate  your  own  understanding  of  each  saying  from  the  explanation  given.  ” 

IN  THE  SOUP:  Stuck.  Not  able  to  escape.  Can 't  get  away. 

1  2  3  4  5 

I  understand:  very  well  don ’t  understand 


Training  for  learning  the  meanings  of  new  similarities  and  for  analogies  was 
accomplished  using  pairs  of  nonsense  words.  Each  of  10,  two-word  sets  was  explained.  Later, 
each  pairing  of  words  was  used  to  test  for  the  similarity  between  the  words  and  how  the  same 
words  fit  into  an  analogy.  The  training  instructions  and  one  example  follow: 
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“On  the  following  pages  you  will  see  how  simple  words  from  rare  languages  are  explained  in 
English...  Carefully  read  the  explanation  for  each  set  of  words.  First  circle  either  a  or  b  for 
the  question  given  after  the  explanation.  Then  circle  the  number  from  1  to  5  that  indicates 
how  well  you,  yourself  now  understand  the  meanings  of  the  words  from  the  explanation. 
Please  rate  your  own  understanding  of  each  set  of  words  from  the  explanation  given.  ” 

BR1LLIG  and  CIDY:  a  BRILLIG  is  easily  picked  from  a  low  branch  and  a  CIDY from  off  a 
vine.  Both  a  BRILLIG  and  a  CIDY  are  juicy  and  delicious. 

BRILLIG  and  CIDY  are  good  to  a.  Eat  b.  Swim  in 

1  2  3  4  5 

I  understand:  very  well  don  V  understand 


Following  training,  the  students  handed  in  their  training  materials  and  then  received  a  set 
of  tests  on  the  newly  learned  material.  Specifically,  to  test  new  knowledge  of  the  seven  word 
meanings,  the  student  was  asked  to: 


“ Circle  the  letter  (a,  b,  c,  d)  next  to  the  word  that  you  think  is  the  correct  definition  of  the 
term.  Do  the  best  you  can  and  give  an  answer  for  every  question.  ” 

BEZANT  a.  hotel  b.  coin  c.  mill  d.  harbor 


To  test  new  knowledge  for  the  sayings,  the  student  was  told: 

“We  would  like  to  find  out  what  people  know  about  the  meanings  of  sayings.  ”  For  example: 
AN  APPLE  A  DAY  KEEPS  THE  DOCTOR  AWAY  means 
a.  Eating  good  food  helps  you  to  stay  healthy  b.  Pay  your  debts 

“ That 's  right.  The  answer  is  a.  Eating  good  food  helps  you  to  stay  healthy.  Here  are  eight 
such  sayings.  Please  circle  what  you  believe  to  be  the  correct  answer  for  each  question.  ” 

IN  THE  SOUP  means  a.  Broke  b.  Rich  c.  Trapped  d.  Knowing 
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To  test  for  knowledge  of  newly  learned  analogies  and  similarities,  the  student  was  told: 


"On  the  following  pages  we  will  be  asking  how  one  pair  of  words  is  similar  to  another  pair  of 
words.  You  will  be  putting  a  circle  around  the  letter  in  front  of  the  correct  answer.  "  For 
example: 

HIGH  is  to  TALL  as  LOW  is  to _  a.  SHORT  b.  RED 

"That 's  right!  The  answer  is  SHORT.  On  the  following  pages  you  will  see  10  pairings  that 
have  to  do  with  how  one  pair  of  words  is  most  like  another  pair  of  words.  Please  circle  the 
letter  you  consider  to  be  the  correct  answer.  Also  on  the  following  pages,  we  will  be  asking 
how  one  word  is  most  like  another  word.  You  will  be  putting  a  circle  around  the  letter  in  front 
of  the  correct  answer.  ”  For  example: 

MAN  and  WOMAN  a.  HUMAN  b.  LIVING  c.  AMERICAN 

"The  answer  is  a.  HUMAN.  Why?  Because  HUMAN  is  the  way  in  which  MAN  and  WOMAN 
are  MOST  alike.  Don  7  skip  any  questions.  If  you  are  not  sure,  make  your  best  guess.  ” 


On  the  following  pages,  students  saw  10  pairs  of  questions  such  as: 


"BRILLIG  is  to  C1DY  as  TREE  is  to 

BRILLIG  and  CIDY 

a.  Grass 

a.  Round 

b.  Vine 

b.  Colored 

c.  Tea 

c.  Fruit 

d.  Ocean 

d.  Vitamins" 

Questions  of  the  sort  traditionally  tested  on  the  SAT-V  were  taken  from  practice  tests  for 
the  SAT-V  (Robinson  &  Katzman,  2003)  and  the  GRE  (Martinson,  2000)  to  comprise  what  we 
will  refer  to  as  the  brief  SAT.  The  test  included  24  items,  8  of  which  tested  knowledge  of  the 
meanings  of  words.  For  example: 


"Directions:  Each  of  the  questions  below  contains  one  or  more  blank  spaces,  each  space 
indicating  an  omitted  word.  Each  sentence  is  followed  by  five  words  or  sets  of  words.  Read 
and  determine  the  general  sense  of  the  sentence.  Then  chose  the  word,  or  set  of  words  that, 
when  inserted  in  the  sentence,  best  fits  the  meaning  of  the  sentence.  " 

Henry  viewed  Melissa  as _ ;  she  seemed  to  be  against  any  position  regardless  of  its 

merits. 

a.  heretical  b.  disobedient  c.  contrary  d.  inattentive  e.  harried 
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A  second  set  of  eight  questions  tested  knowledge  of  opposites.  For  example: 


" Directions :  Each  of  the  following  questions  consists  of  a  word  printed  in  capital  letters, 
followed  by  five  words  or  phrases.  Choose  the  word  or  phrase  that  is  most  nearly  opposite  in 
meaning  to  the  word  in  capital  letters.  Consider  all  the  choices  before  deciding  which  one  is 
best.  ” 

EXONERATE  a.  testify  b.  engender  c.  accuse  d.  inundate  e.  abrogate 


A  third  set  of  eight  questions  tested  knowledge  of  analogies.  For  example: 


“Directions:  In  each  of  the  following  questions,  you  are  given  a  related  pair  of  words  in 
capital  letters.  Each  capitalized  pair  is  followed  by  five  (5)  pairs  of  words.  Choose  the  pair 
that  best  expresses  a  relationship  similar  to  that  expressed  by  the  original  pair.  " 

WATERFALL:  CASCADE:: 

a.  snow:  freeze  b.  missile:  launch  c.  tree:  exfoliate  d.  wave:  undulate  e.  monarch:  reign 


Time 


An  entire  session,  including  training  and  testing,  lasted  about  35  minutes. 


Results 


Data  on  the  acquisition  of  new  knowledge  was  based  on  the  student’s  total  score  across 
tests  of  word  meaning,  meanings  of  sayings,  knowledge  of  similarities,  and  knowledge  of 
analogies.  Total  scores  based  on  the  brief  SAT  type  tests  of  knowledge  of  meanings,  opposites, 
and  analogies  described  above  were  the  second  variable  of  interest.  Finally,  numerical  grades  in 
class  constituted  the  measure  of  achievement. 

The  initial  question  was  whether,  in  accord  with  earlier  findings  (Fagan  &  Holland  2002, 
2007),  the  minority  group  members  and  the  majority  group  members  in  the  present  research 
differed  in  performance  on  standard  indices  (SAT-type  questions  and  class  grades)  which  rely  on 
past  knowledge  of  word  meanings,  but  did  not  differ  in  knowledge  of  newly  learned  material.  Of 
the  48  minority  members  in  the  sample,  42  were  students  at  one  private  university.  These  42 
students  were  compared  with  the  295  remaining  students  from  that  same  university  for  new 
learning,  SAT-type  performance,  and  academic  achievement.  Table  1  lists  the  mean  scores  for 
each  measure  for  each  racial  group  along  with  the  t  values  resulting  from  comparisons  of  the 
racial  groups  on  each  variable. 
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Table  1 

New  Knowledge,  Brief  SA  T,  and  Grades  for  Racial  Majority  and  Minority  Group  Members  and 
Associated  t  Values 


Race 

Mean 

SD 

t 

P 

New  Knowledge 

Brief  SAT 

Class  Grades 

Majority 

Minority 

Majority 

Minority 

Majority 

Minority 

26.9 

27.0 

15.8 

14.3 

84.3 

77.9 

5.1 

4.4 

4.3 

5.1 

9.0 

14.2 

0.2 

2.1 

3.9 

n.s. 

<.05 

<.001 

The  present  findings  are  in  accord  with  earlier  results.  Minority  and  majority  group 
members  did  not  differ  in  knowledge  when  equal  opportunity  for  previous  exposure  to 
information  was  experimentally  controlled  (tests  of  new  knowledge).  They  did  differ  when 
previous  exposure  to  information  was  not  controlled  (brief  SAT  and  class  grades).  For  our 
present  purposes,  the  results  in  Table  1  confirm  the  culture- fair  nature  of  the  items  chosen  to 
measure  new  knowledge. 

The  next  question  is  what  are  the  relationships  among  the  indices  of  new  knowledge, 
SAT-type  scores  and  class  grades?  In  determining  these  relationships,  correlations  were 
computed  among  variables.  Both  the  uncorrected  correlations  and  the  correlations  corrected  for 
unreliability  were  reported.  The  means  and  standard  deviations  for  the  test  of  new  learning,  the 
brief  SAT  test,  and  class  grade  along  with  estimates  of  reliability  based  on  Kuder-Richardson 
formula  21,  (Cronbach,  1960)  are  listed  in  Table  2  for  the  total  sample  of  484  participants. 

Table  2 

Estimates  of  Reliability  for  New  Knowledge,  Brief  SAT  Scores,  and  Grades 


Mean 

SD 

%  correct 

Reliability 

New  Knowledge 

25.6 

5.7 

73.1 

.81 

Brief  SAT 

14.2 

5.1 

59.2 

.81 

Class  Grade 

83.4 

8.2 

83.4 

.86 

Note.  N  =  484. 

Table  3  lists  the  obtained  coefficients,  both  uncorrected  and  corrected  for  attenuation  due 
to  unreliability,  for  the  total  sample  of  484  participants  in  the  initial  effort. 
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Table  3 

Correlations  Among  New  Knowledge,  Brief  SAT,  and  Grades,  Uncorrected  (r)  and  Corrected 
(R)  for  Unreliability 


Brief  SAT 

Grades 

New  Knowledge 

r  .58* 

R  .72* 

r  .19* 

R  .23* 

Brief  SAT 

r  .29* 

R  .35* 

Note.  N  =  484,  *p  <  .01 . 

The  most  relevant  information  to  note  from  the  data  given  in  Table  3  is  that  scholastic 
achievement  was  predicted  at  a  low  but  statistically  significant  level  by  a  culture- fair  test  of  new 
knowledge  and  that  there  was  a  substantial  and  significant  relationship  between  the  culture-fair 
test  of  new  knowledge  and  the  Brief  SAT  index.  The  coefficients  obtained  between  grades  and 
new  knowledge  noted  in  Table  3,  however,  may  be  attenuated  in  at  least  three  ways.  The  first  is 
that  the  35-item  test  of  new  learning  proved  to  be  quite  easy  at  73.1%  correct  as  compared  to 
59.2%  correct  for  the  brief  SAT.  A  more  difficult  test  of  new  learning  might  result  in  better 
predictive  validity.  The  second  fact  contributing  to  attenuation  is  variability  in  grading  from 
school  to  school  or  from  class  to  class  within  a  school  (i.e.,  teachers  vary  in  how  they  grade).  The 
third  fact  is  that  the  majority  of  the  participants  in  Experiment  1  (70%)  were  drawn  from  one 
private  university  with  very  high  admission  standards.  Among  the  total  sample  in  Experiment  1, 
however,  we  were  fortunate  to  obtain  grades  from  the  same  teacher  who  taught  the  same  course 
at  both  a  community  college  and  a  4-year  college.  The  two  classes  included  68  students.  Table  4 
presents  the  means  and  standard  deviations  for  the  tests  of  new  learning,  the  brief  SAT  tests,  and 
class  grades  for  those  68  participants  along  with  Kuder-Richardson  reliabilities  for  each  measure. 


Table  4 

Estimates  of  Reliability,  Means,  and  SDs  for  New  Knowledge,  Brief  SAT  Scores,  and  Grades 


Mean 

SD 

%  correct 

Reliability 

New  Knowledge 

23.0 

5.6 

66.0 

.77 

SAT 

10.3 

4.5 

43.0 

.74 

Class  Grade 

84.4 

8.2 

84.4 

.81 

Note.  N  =  68. 

The  coefficients,  uncorrected  and  corrected  for  unreliability,  between  scores  on  the 
culture-fair  test  of  new  knowledge,  the  brief  SAT  items,  and  class  grades  for  these  68  students 
are  presented  in  Table  5. 
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Table  5 

Correlations,  Uncorrected  (r)  and  Corrected  (R)  for  Unreliability  Among  New  Knowledge,  Brief 
SA  T  Scores,  and  Grades 


Brief  SAT 

Grades 

New  Knowledge 

r  .58* 

R  .77* 

r  .41* 

R  .52* 

Brief  SAT 

r  .55* 

R  .70* 

Note.  N  =  68,  *p<.01. 

As  one  can  see,  when  a  common  metric  for  grades  is  available  and  range  of  ability  is 
increased,  the  relationship  between  the  test  of  new  knowledge  and  either  the  brief  SAT  test  or 
performance  in  class  is  substantial.  One  might  also  note  that  the  relationships  noted  in  Tables  3 
and  5  between  the  brief  SAT  and  class  grades  are  higher  than  the  relationships  between  new 
learning  and  class  grades.  Such  a  disparity  is  to  be  expected.  Processing  ability  (measured  here 
by  the  new  learning  task)  plays  a  role  in  how  much  knowledge  is  gained  over  time,  hence  new 
knowledge  predicts  both  brief  SAT  scores  and  grades.  The  brief  SAT  scores,  however,  are  based 
not  only  on  processing  ability  but  on  specific  information  provided  by  one’s  culture.  Since 
information  provided  by  one’s  culture — information  necessary  to  solve  items  on  the  brief  SAT 
(e.g.,  knowledge  of  word  meanings) — also  plays  a  role  in  understanding  course  material;  one 
would  expect  the  brief  SAT  scores  to  be  somewhat  better  predictors  of  grades  than  the  new 
learning  scores. 

Finally,  in  employing  the  brief  SAT  test  constructed  for  the  present  effort,  we  wanted  a 
measure  comparable  to  the  standard  SAT-V  as  to  predictive  validity  for  scholastic  achievement. 
A  subset  of  294  of  the  484  students  who  participated  in  Experiment  1  attended  the  same  private 
university.  These  294  students  had  taken  the  SAT  in  2003-2004  allowing  a  direct  comparison 
between  the  brief  SAT  constructed  for  the  present  research  and  conventional  SAT-V  scores.  The 
two  estimates  were  themselves  highly  correlated  at  r  =  .73  with  a  correlation  corrected  for 
unreliability  (R)  of  .86.  Table  6  describes  how  these  294  students  performed  on  the  SAT-V  and 
the  brief  SAT  test  and  class  grades  along  with  Kuder-Richardson  21  reliabilities  for  each 
measure. 
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Table  6 

SAT-V  Scores,  Brief  SAT  Scores,  Grades,  and  Reliability  Estimates 


Mean 

SD 

Reliability 

SAT-V 

630.9 

86.1 

.92 

Brief  SAT 

15.4 

4.7 

.78 

Grades 

83.5 

9.4 

.85 

Note.  N  =  294. 

As  one  can  see  from  the  data  in  Table  6,  the  students,  as  a  group,  scored  highly  both  in 
academic  (83.5%)  and  in  conventional  aptitude  test  performance  (630.9  on  the  SAT-V). 

How  well  did  the  SAT-V  and  the  brief  SAT  test  predict  academic  achievement?  The  predictive 
validity  coefficients,  corrected  ( R )  or  not  corrected  (r)  for  unreliability,  are  listed  in  Table  7. 

Table  7 

Prediction  of  Grades  on  the  Basis  of  the  SA  T-  V  and  the  Brief  SA  T,  Uncorrected  (r)  and 
Corrected  (R)  for  Unreliability 


Grade 

SAT-V 

r  =  .28* 

R  =  .30* 

Brief  SAT 

r  =  .25* 

fl  =  .31* 

Note.  N  =  294,  *  p  <  .01 . 

As  is  evident  from  the  data  listed  in  Table  7,  both  the  SAT-V  and  the  brief  SAT  test  were 
successful  in  predicting  academic  performance.  All  values  were  significantly  greater  than  chance 
and  of  moderate  scope.  Of  most  importance  is  the  fact  that  the  predictions  from  each  test  to 
academic  performance  were  virtually  identical.  No  significant  differences  were  found  between 
the  SAT-V  predictions  of  class  grade  or  the  brief  SAT  test’s  prediction  of  class  grade.  Whether 
such  similarities  in  prediction  may  be  influenced  by  restriction  of  range  remains  a  question  for 
further  study. 


Experiment  2 

The  results  of  Experiment  1  indicated  that  the  tests  of  new  learning,  developed  to  that 
point,  were  culture- fair,  reliable,  and  predictive  of  both  numerical  grades  in  class  and  of 
performance  on  tests  akin  to  standard  tests  of  scholastic  aptitude  (brief  SAT  sores).  Experiment  2 
involved  a  much  higher  percentage  of  community  college  students  relative  to  those  in  private 
universities  and  colleges  than  was  possible  for  Experiment  1.  Such  sampling  allowed  a  more 
representative  estimate  of  performance  on  new  learning  tests,  the  brief  SAT,  and  academic 
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achievement.  In  the  construction  of  any  test,  some  items  are  found  to  be  more  predictive  than 
others.  We  analyzed  the  data  from  Experiment  1  to  determine  which  items  from  the  culture-fair 
test  of  new  learning  would  be  most  predictive  and  of  moderate  difficulty  level.  We  retained  those 
items  in  a  revised  version  of  the  test  and  added  additional,  new  items  which  appeared  to  be  of 
equal  difficulty  to  the  selected  items.  These  modifications  to  the  sample  and  test  characteristics 
allowed  a  more  accurate  determination  of  the  reliability  and  predictive  validity  of  the  culture-fair 
test  of  new  learning. 

Participants 

The  sample  included  696  students  (425  females,  271  males).  Those  of  majority  ethnic 
status  constituted  78%  of  the  sample  and  those  of  minority  ethnic  status  22%.  The  mean  age  of 
the  participants  was  21.2  years  (SD  5.8  years).  The  mean  education  was  13.7  years  (SD  1.3 
years).  Some  52%  of  the  students  were  enrolled  at  private  universities  (40%  at  one  and  1 2  %  at 
another),  2%  attended  a  local  college,  and  45%  attended  a  two-year  community  college.  All 
participants  were  registered  for  psychology  classes.  Specific  demographic  characteristics  were 
again  gathered  by  asking  students  to  complete  a  form  that  asked  for  their  age,  gender,  and  years 
of  education.  Racial  identity  was  again  determined  by  the  students  who  filled  out  the  United 
States  Public  Health  Service  form  noted  earlier. 

Materials 

Participants  were  given  the  same,  24-item,  brief  SAT  test  used  in  Experiment  1 .  Standard 
SAT-V  scores  were  available  for  213  of  the  students  at  the  participating  universities.  Measures 
of  specific  academic  achievement  (exam  performance  on  objective  tests)  were  obtained  from 
instructors  of  Psychology  courses  who  were  asked  to  provide  the  students’  numerical  scores  on 
the  tests  in  the  course.  An  extended,  48-item  culture-fair  test  of  the  students’  ability  to  acquire 
new  information  concerning  the  meanings  of  previously  unknown  words,  sayings,  similarities, 
and  analogies  was  given  in  Experiment  2  and  will  be  noted  below. 

Procedure 

As  before,  the  tests  of  new  knowledge  were  based  on  a  training  phase  and  a  testing  phase 
with  all  training  and  testing  done  in  a  group  setting  during  set-aside  class  time.  Training  for  the 
learning  of  the  meanings  of  new  words,  sayings,  similarities,  and  analogies  followed  the  same 
format  used  in  Experiment  1.  In  Experiment  2,  participants  learned  the  meanings  of  12  new 
words  and  16  new  sayings.  Training  for  learning  the  meanings  of  new  similarities  and  for 
analogies  was  accomplished  using  20  pairs  of  nonsense  words.  Following  training,  the  students 
handed  in  their  training  materials  and  then  received  a  set  of  tests  on  the  newly  learned  material. 
The  tests  for  how  well  the  meanings  of  new  words  and  sayings  were  learned  followed  the  same 
format  as  employed  in  Experiment  1 .  The  testing  for  how  newly  learned  nonsense  words  could 
be  used  in  solving  similarities  and  analogies  differed  from  the  testing  procedure  employed  in 
Experiment  1 .  In  Experiment  2,  1 0  pairs  of  nonsense  words  of  the  20  that  had  been  learned  were 
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used  to  test  knowledge  of  analogies.  An  entire  session  (training  and  testing)  lasted  about  45 
minutes. 

Results 


As  in  Experiment  1,  data  on  the  acquisition  of  new  knowledge  was  based  on  the  students’ 
total  score  across  tests  of  newly  acquired  meanings  of  words,  sayings,  similarities,  and  analogies. 
Total  score  based  on  the  brief  SAT-type  tests  of  knowledge  of  meanings,  opposites,  and 
analogies  was  the  second  variable  of  interest  and  numerical  grades  in  class  constituted  the 
measure  of  achievement. 

The  first  focus  was  on  questions  concerning  the  role  of  racial-ethnic  minority  status  in  the 
relationships  among  knowledge  of  newly  learned  material,  past  knowledge  as  estimated  by  the 
brief  SAT  scores,  and  grades  in  class.  These  questions  were  explored  in  a  series  of  multiple 
regression  analyses.  The  first  two  of  these  analyses  were  based  on  692  participants  for  whom 
scores  on  new  learning  and  the  brief  SAT  were  available  and  who  had  indicated  their  ethnic 
status.  According  to  the  theory  guiding  the  present  research,  one  would  expect  both  racial-ethnic 
status  and  new  learning  ability  to  play  a  role  in  past  knowledge  as  reflected  in  brief  SAT  scores. 
Indeed,  such  was  the  case.  The  regression  analysis  yielded  a  multiple  R  of  .64,  F  (2,690)  =  236.6, 
p  <  .001 ,  with  Beta  values  of  .08  ( t  =  2.7,  p  <  .01)  and  .62  {t  =  21 .0,  p  <  .001)  for  minority  status 
and  new  learning,  respectively,  for  the  prediction  of  brief  SAT  scores.  Most  importantly  for  our 
present  purposes,  however,  one  would  expect  past  knowledge  (i.e.,  brief  SAT  scores),  but  not 
racial-ethnic  status  to  be  related  to  the  ability  to  process  new  information.  The  results  were  as 
expected.  The  regression  analysis  yielded  a  multiple  R  of  .63,  F  (2,690)  =  23 1 .3,  p  <  .001 ,  with 
Beta  values  of  .03  (/  =  0.9,  n.s.)  and  .63  (/  =  21.0,/?  <  .001)  for  minority  status  and  brief  SAT 
scores,  respectively,  for  the  prediction  of  new  learning. 

A  similar  analysis  was  undertaken  for  the  contributions  of  racial-ethnic  status  and  new 
learning  ability  to  academic  achievement.  These  analyses  were  based  on  593  participants  for 
whom  scores  on  new  learning  and  class  grades  were  available  and  who  had  indicated  their  ethnic 
status.  Again,  one  would  expect  both  racial-ethnic  status  and  new  learning  ability  to  play  a  role 
in  academic  achievement.  They  did,  with  a  multiple  R  of  .36,  F  (2,591)  =  44.4,  p  <  .001,  with 
Beta  values  of  .08  ( t  =  2.0,/?  <  .05)  and  .35  (t  =  9.7,  p  <  .001)  for  minority  status  and  new 
learning,  respectively,  for  the  prediction  of  grades.  Achievement  and  new  learning  ability  would 
undoubtedly  be  related.  But  does  racial-ethnic  status  contribute  any  independent  variance  to  the 
prediction  of  new  learning  ability  once  the  association  between  new  learning  ability  and 
achievement  is  taken  into  account?  No,  a  regression  analysis  incorporating  racial-ethnic  status 
and  achievement  as  predictors  of  new  learning  ability  yielded  a  multiple  R  of  .35,  F  (2,591 )  = 
42.4,/?  <  .001,  with  Beta  values  of  .03  (i  =  0.7,  n.s.)  and  .35  0  =  9.1,/?  <  .001)  for  minority  status 
and  grades,  respectively,  for  the  prediction  of  new  learning. 

The  present  findings  are  in  accord  with  the  results  of  Experiment  1  where  minority  and 
majority  group  members  did  not  differ  in  knowledge  when  equal  opportunity  for  previous 
exposure  to  information  was  experimentally  controlled  (tests  of  new  knowledge).  The  groups  of 
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different  racial-ethnic  background  did  differ  when  previous  exposure  to  information  was  not 
controlled  (brief  SAT  and  class  grades).  For  our  present  purposes,  the  results  of  both  experiments 
confirm  the  culture-fair  nature  of  the  items  chosen  to  measure  new  knowledge. 

The  relationships  among  the  indices  of  new  knowledge,  SAT-type  scores,  and  class 
grades,  both  uncorrected  and  corrected  for  unreliability,  for  596  participants  in  Experiment  2  will 
now  be  considered.  Table  8  lists  means  and  standard  deviations  for  the  test  of  new  learning,  the 
brief  SAT  test,  and  class  grades  along  with  estimates  of  reliability  based  on  Kuder-Richardson 
formula  2 1 . 

Table  8 

Estimates  of  Reliability  for  New  Knowledge,  Brief  SAT  Scores,  and  Grades 


Mean 

SD 

%  correct 

Reliability 

New  Knowledge 

25.5 

7.1 

53.2 

.78 

Brief  SAT 

11.6 

5.3 

48.4 

.82 

Class  Grade 

81.0 

12.2 

81.0 

.90 

Note.  N  =  596. 

Testing  in  Experiment  2  involved  a  much  higher  percentage  of  community  college 
students  relative  to  those  in  private  universities  and  colleges  than  was  possible  for  Experiment  1 , 
allowing  a  more  representative  estimate  of  performance  on  new  learning  tests,  the  brief  SAT,  and 
academic  achievement.  The  sampling  procedures  in  Experiment  2  succeeded  in  providing  greater 
variability  in  scores  for  new  knowledge,  the  brief  SAT,  and  grades  than  was  the  case  in 
Experiment  1 .  Similarly,  the  addition  of  items  to  the  test  of  new  knowledge  in  Experiment  2 
resulted  in  scores  similar  in  difficulty  to  the  brief  SAT  scores.  These  modifications  to  sample  and 
test  characteristics  allowed  a  more  accurate  determination  of  the  reliability  and  predictive 
validity  of  the  culture-fair  test  of  new  learning.  The  obtained  coefficients,  either  uncorrected  or 
corrected  for  attenuation  due  to  unreliability,  for  the  total  sample  of  596  participants  are  listed  in 
Table  9. 

Table  9 


Correlations  Among  New  Knowledge,  Brief  SAT,  and  Grades,  Uncorrected  (r)  and  Corrected 
(R)  for  Unreliability 


Brief  SAT 

Grades 

New  Knowledge 

r  .63* 

R  .79* 

r 

R 

.36* 

.43* 

Brief  SAT 

r 

.46* 

R 

.53* 

Note.  N  =  596,  *  p  <  .01 . 
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The  most  relevant  information  to  note  from  the  data  given  in  Table  9  is  that  scholastic 
achievement  is  predicted  at  a  moderate  and  statistically  significant  level  by  a  culture-fair  test  of 
new  knowledge  and  there  is  a  substantial  and  significant  relationship  between  the  culture-fair  test 
of  new  knowledge  and  the  brief  SAT  index.  The  coefficients  obtained  between  grades  and  new 
knowledge  noted  in  Table  9  are  considerably  greater  than  those  obtained  in  Experiment  1, 
because  a  more  difficult  task  of  new  learning  ability  was  employed  in  Experiment  2  and  the 
participants  in  Experiment  2  were  drawn  from  a  more  representative  sample  of  institutions  of 
higher  education  than  was  the  case  in  Experiment  1.  Variability  in  grading  from  school  to  school 
and  from  class  to  class  within  a  school  remained  as  an  attenuating  factor  in  assessing  the 
accuracy  of  predictive  validity  coefficients.  As  noted  in  the  analyses  of  Experiment  1,  when  a 
common  metric  for  grades  is  available  (the  same  teacher  for  the  same  course  in  both  a  college 
and  a  community  college)  and  range  of  ability  is  increased,  the  relationship  between  the  culture- 
fair  test  of  new  knowledge  and  either  the  brief  SAT  test  or  performance  in  class  is  substantial. 

As  also  noted  in  Experiment  1 ,  and  again  in  Experiment  2,  the  relationships  between  the 
brief  SAT  and  class  grades  are  somewhat  higher  than  the  relationships  between  new  learning  and 
class  grades.  Again,  this  disparity  is  to  be  expected  because  learning  ability  plays  a  role  in 
knowledge  acquisition  and,  thus,  predicts  both  brief  SAT  scores  and  grades.  However,  the  past 
information  provided  by  one’s  culture  also  determines  performance  on  the  brief  SAT  and  the 
understanding  of  course  material.  Hence,  the  brief  SAT  scores  are  somewhat  better  predictors  of 
grades  than  the  new  learning  scores  alone. 

A  further  multiple  regression  analysis  assessed  if  the  test  of  new  knowledge,  being  highly 
correlated  with  the  brief  SAT,  would  add  variance  to  the  prediction  of  grades,  variance 
independent  of  that  predicted  by  the  brief  SAT  scores.  The  test  of  new  learning  did  add 
independent  variance  to  the  prediction  of  grades  as  indicated  by  a  multiple  R  of  .47,  F  (2,593)  = 
82.9,  p  <  .001,  with  Beta  values  of  .11  (t  =  2.4,  p  <  .02)  and  .39  (t  =  8.3,p<  .001)  for  new 
learning  and  the  brief  SAT,  respectively,  for  the  prediction  of  grades. 

Experiment  2  also  allowed  a  further  estimate  of  whether  the  brief  SAT  employed  in  the 
present  experiments  was  comparable  to  the  standard  SAT-V  as  to  predictive  validity  for 
scholastic  achievement.  Experiment  2  included  213  students  attending  two  private  universities 
who  had  taken  the  SATs  in  2005-2006.  The  213  students  included  109  males,  104  females,  189 
racial-ethnic  majority  group  members,  and  24  racial-ethnic  minority  group  members  with  an 
average  age  of  19.3  years,  SD  1.6.  Table  10  presents  a  description  of  how  these  students 
performed  on  the  SAT-V  and  the  brief  SAT  test  along  with  their  class  numerical  grades. 
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Table  10 

SAT-V  Scores,  Brief  SAT  Scores,  Grades,  and  Reliability  Estimates 


Mean 

SD 

Reliability 

SAT-V 

610.8 

87.2 

.98 

Brief  SAT 

14.8 

4.4 

.74 

Grades 

85.2 

9.3 

.86 

Note.  N  =  213. 

The  brief  SAT  and  the  SAT-V  scores  were  themselves  again  correlated  at  r  =  .75  with  a 
correlation  corrected  for  unreliability  ( R )  of  .88.  Again,  the  main  focus  of  the  analysis  was  on 
how  well  the  SAT-V  and  the  brief  SAT  test  predicted  academic  achievement.  The  predictive 
validity  coefficients,  corrected  ( R )  and  not  corrected  (r)  for  unreliability,  for  the  participants  in 
Experiment  2  are  listed  in  Table  1 1 . 

Table  1 1 

Prediction  of  Grades  on  the  Basis  of  the  SA  T-  V  and  the  Brief  SA  T,  Uncorrected  (r)  and 
Corrected  (R)  for  Unreliability _ _ 

Class  Grade _ 

r  =  .35* 

R  =  .38* 
r  =  .33* 

R  =  A\* 


SAT-V 
Brief  SAT 


Note.  N  =  213,  *  p<  .01. 

As  you  can  see  from  the  data  in  Table  1 1,  both  the  SAT-V  and  the  Brief  SAT  test  were 
again  successful  in  predicting  academic  performance.  Once  more,  the  predictions  from  each  test 
to  academic  performance  were  virtually  identical.  No  significant  differences  (via  t  tests)  were 
found  between  the  SAT-V  predictions  of  class  grade  or  the  brief  SAT  test’s  prediction  of  class 
grade. 

DISCUSSION 

Summary 

The  present  research  addressed  the  question  of  whether  a  racially  unbiased  test  of  the 
ability  to  process  information  would  predict  how  well  young  adults  succeed  in  college  classes. 
Current  theories  of  intelligence  are  based  on  an  assumption  that  all  those  taking  IQ  tests  have  had 
equal  opportunity  for  exposure  to  the  information  being  tested.  Thus,  past  efforts  to  develop  an 
intelligence  test  that  is  culture-fair  have  not  been  successful.  The  present  research  is  based  on  a 
theory  (Fagan,  1992,  2000)  that  assumes  that  we  act  on  the  basis  of  what  we  know  and  that  what 
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we  know  is  a  result  of  our  processing  of  the  information  provided  to  us  by  our  culture.  Based  on 
these  assumptions,  the  IQ  score  is  seen  as  a  measure  of  knowledge.  Specifically,  how  much  you 
know  (your  IQ)  depends  on  how  well  you  process  information  (your  intelligence)  and  also  on  the 
information  given  to  you  by  your  culture  to  process.  The  theory  has  been  used  to  explain  the 
source  of  IQ  differences  between  Americans  of  different  races.  Specifically,  our  past  research 
(Fagan  &  Holland,  2002,  2007)  has  found  that  American  Whites  and  African  Americans  who 
differ  in  IQ  do  not  differ  in  their  ability  to  process  information  when  that  information  has  been 
equally  accessible  to  both  groups.  In  the  present  experiments,  college  students  were  given 
multiple-choice  tests  of  their  ability  to  acquire  new  information  concerning  the  meanings  of 
previously  unknown  words,  sayings,  similarities,  and  analogies.  They  also  were  tested  for  their 
knowledge  of  vocabulary,  opposites,  and  analogies  with  a  brief  version  of  the  SAT-V 
constructed  for  the  present  experiments.  The  brief  SAT  provided  a  standard  measure  of  academic 
aptitude  that  would  be  common  to  both  community  college  students  and  university  students. 
Numerical  grades  in  courses  were  obtained  for  the  majority  of  participants.  Associations  among 
performance  on  the  culture-fair  tests  of  new  learning,  academic  aptitude  (the  brief  version  of  the 
Scholastic  Assessment  Test  scores),  and  specific  achievement  (objective  test  scores  in  college 
courses)  were  analyzed. 

The  findings  reveal  that  tests  of  new  learning  are  culture-fair,  reliable,  and  predictive  of 
both  academic  achievement  (numerical  grades  in  class)  and  of  a  brief  version  of  a  standard  test 
of  scholastic  aptitude  (the  Scholastic  Assessment  Test-Verbal).  Further,  the  results  demonstrate 
that  tests  of  new  learning  and  tests  of  existing  knowledge  (such  as  the  brief  SAT)  each  contribute 
independent  variance  to  the  prediction  of  class  grades.  A  final  notable  finding  is  that  the  brief, 
24-item  version  of  the  SAT-V  that  was  created  for  the  present  investigation  turned  out  to  be  as 
predictive  of  college  grades  as  the  standard  SAT-V.  In  summary,  the  present  research  indicates 
there  are  at  least  two  factors  that  determine  success  in  a  complex  learning  environment  such  as 
higher  education:  one  is  information  processing  (new  learning)  ability  and  the  other  is  the  extent 
of  knowledge  one  has  acquired  to  that  point. 

Theoretical  Significance 

Jensen  (1998)  believes  that  the  IQ  score  reflects  what  he  calls  the  general  factor  (or  g) 
underlying  intelligence.  Differences  in  IQ  scores  are  due  to  differences  ing.  According  to 
Jensen,  genetic  factors  play  a  primary  role  in  determining  g.  Thus,  Jensen  (2000)  believes  that  it 
may  not  be  possible  to  come  up  with  tasks  that  show  no  differences  in  test  performance  between 
African  Americans  and  Whites  and  yet  still  predict  academic  performance.  Fagan  (2000)  does 
not  agree  with  Jensen,  pointing  out  that  the  failure  to  develop  tests  of  intelligence  that  can  be 
fairly  applied  across  racial  groups  stems  from  a  theoretical  bias  to  equate  the  IQ  score  with 
intelligence  rather  than  with  knowledge.  If  intelligence  is  defined  as  information  processing  and 
the  IQ  score  as  knowledge,  the  possibility  of  culture-fair  tests  of  intelligence  based  on  estimates 
of  information  processing  arises. 
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The  chief  theoretical  implication  of  the  results  of  the  present  research  is  that  the 
assumption  of  equal  opportunity  for  exposure  to  information  made  by  theories  of  intelligence  is 
false.  When  such  an  assumption  is  shown  to  be  false,  it  becomes  possible  to  develop  culture-fair 
tests  of  intelligence  that  allow  basic  intellectual  abilities  to  be  recognized.  The  present  findings 
and  those  of  Fagan  and  Holland  (2002, 2007)  are  consistent  with  other  studies  that  have 
attempted  to  provide  equal  opportunity  for  exposure  to  information  to  people  of  different  races. 

In  an  early  study,  Bridgeman  and  Buttram  ( 1 975)  found  that  training  verbal  strategies  erased  the 
differences  between  African  American  and  White  school  children  on  tests  of  analogy  solutions. 
More  recently,  Sternberg  et  al.  (2002)  showed  that  teaching  cognitive  skills  and  strategies  to 
African  children  in  Tanzania  increased  their  scores  (relative  to  children  not  so  trained)  on  tests  of 
syllogisms,  sorting,  and  twenty  questions.  Finally,  Skuy  et  al.  (2002)  found  that  Black  college 
students  in  South  Africa  given  a  mediated  learning  experience  reaped  significantly  more  benefit 
from  such  training  on  tests  of  matrix  solution  than  did  similarly  trained  White  peers. 

In  brief,  the  present  experiments  serve  as  an  example  of  how  a  long-lived  and  currently 
much  debated  issue  such  as  culture-fair  tests  of  intelligence  can  be  addressed  by  a  theory  which 
defines  intelligence  as  information  processing  and  by  experimental  studies  guided  by  such  a 
theory.  The  present  results  also  contribute  to  two  main  theoretical  goals  of  the  ARI  basic 
research  program:  to  discover  relationships  among  information  processing  ability  and  knowledge 
that  lead  to  achievement  and  productivity  in  complex  situations  and  to  demonstrate  that 
intelligence  is  a  function  of  an  aptitude  [learning  ability  in  novel  situations]  that  cuts  across 
domains  and  how  to  develop  a  method  for  measuring  this  aptitude  [and  to  further]  validate  [this] 
cognitive  ability  against  relevant  criteria  (BRO-BAA,  page  5,  III,  D,  1 ).  In  future  work  it  also 
might  be  informative  to  conduct  research  on  the  long-term  retention  of  newly  gained  knowledge, 
to  see  if  the  ability  to  retain  new  knowledge  is  as  predictive  as  or  even  more  predictive  of 
relevant  achievement  criteria  than  the  ability  to  encode  new  information.  In  the  present  research, 
achievement  was  based  primarily  on  performance  in  introductory  classes  in  Psychology.  Future 
work  might  investigate  the  role  of  new  learning  ability  on  achievement  in  other  academic 
specialties  and  in  classes  ranging  from  introductory  to  advanced.  Such  studies  would  aid  in 
clarifying  the  relative  impact  of  new  learning  ability  and  prior  exposure  to  information  on 
academic  performance  in  general. 

Practical  Implications 

We  know  that  there  exists  a  sizable  gap  in  SAT  or  ACT  scores  between  students  of 
majority  or  minority  racial-ethnic  status  (Zwick,  2002).  One  solution  to  the  disparity  in 
admission  rates  to  college  due  to  race  is  to  abandon  tests  such  as  the  SAT  or  ACT.  Such  a 
solution  is  impractical  and  is  unlikely  to  be  adopted  by  the  majority  of  institutions  of  higher 
learning.  Another  solution  is  to  develop  culture-fair,  valid  tests  of  academic  aptitude  that  may  be 
used  in  place  of,  or  more  likely,  in  addition  to  conventional  tests  such  as  the  SAT  or  ACT.  Initial 
attempts  at  the  development  of  such  a  culture-fair  test  have  been  met  with  limited  success 
(Sternberg,  2006).  The  present  experiments,  however,  provide  evidence  that  brief,  reliable  tests 
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of  information  processing  based  on  the  ability  to  acquire  new  information  are  valid  predictors  of 
academic  achievement  and  do  not  discriminate  between  groups  differing  in  racial-ethnic  status. 

Two  of  the  guiding  missions  of  ARI,  both  historically  and  currently,  are  to  provide  new 
technology  to  meet  the  personnel  challenges  of  the  Army  in  recruiting,  selecting,  and  assigning 
quality  people  and  to  provide  scientific  insight  into  leader  development.  The  present  research 
provides  basic  scientific  data  to  aid  in  Soldier  selection,  assignment,  and  performance  and  in 
leader  development.  Specifically,  the  results  of  the  present  research  aid  the  Army  in  measuring 
individual  differences  in  cognitive  ability  without  regard  to  racial-ethnic  status.  Such  estimates 
of  cognitive  ability  (based  on  brief,  cost-effective  tasks)  can  be  used  in  the  selection  of  Soldiers 
and  in  the  identification  of  mature,  effective  leaders.  Tests  of  information  processing  ability  that 
are  valid  predictors  of  achievement  in  complex  situations  provide  a  means  to  evaluate  volunteers 
and  select  candidates  for  advanced  education  or  training  in  complex  situations  of  the  sort 
performed  by  Army  personnel.  Such  tests  also  can  be  employed  in  further  selection  and  training 
stages.  In  the  field,  such  skills  are  necessary  to  make  quick  and  correct  decisions  based  on 
rapidly  incoming  information.  Skill  in  information  processing  is  likely  to  be  a  key  indicator  of 
effective  future  Army  leaders.  Finally,  using  tests  of  information  processing  known  to  be  culture- 
fair  may  provide  an  increase  in  the  number  of  eligible  recruits  and  an  incentive  to  re-enlistment 
on  the  part  of  minorities. 

Currently,  questions  have  been  raised  in  a  series  of  articles  in  the  New  York  Times  (e.g., 
Lewin,  2006)  about  the  amount  of  time  necessary  to  complete  the  SAT,  which  is  estimated  at  3 
hours  and  45  minutes  (and  longer  for  students  with  disabilities).  Earlier  reports  in  the  New  York 
Times  (e.g.,  Arenson,  2005)  revealed  a  sizable  error  rate  in  the  current  scoring  process  for  the 
SAT.  The  findings  of  the  present  research  as  to  the  predictive  validity  of  a  brief  SAT  have 
practical  implications  for  the  use  of  lengthy  tests  of  aptitude  in  general.  In  the  present,  ARI- 
sponsored,  research  we  found  that  a  very  brief,  easily  scored  version  of  the  SAT-V  was  as 
predictive  of  college  grades  as  the  standard  SAT-V.  Data  on  predictive  validity  relative  to  test 
length  is  an  interesting  empirical  question  to  pursue  in  future  studies  and  may  be  of  benefit  to  the 
ARI  applied  program  for  Selection,  Classification,  and  Performance  Metrics  for  the  Future  Force 
Soldier  (STO  IV.  SP.  2002.1). 

Conclusions 

In  summary,  the  present  experiments  find  that  there  are  at  least  two  factors  that  determine 
success  in  a  complex  learning  environment  such  as  higher  education;  one  is  information 
processing  (new  learning)  ability  and  the  other  is  the  extent  of  knowledge  one  has  acquired  to 
that  point.  In  an  earlier  article  (Fagan,  2000,  p  177),  the  hope  was  expressed  that  “culture- fair 
intelligence  tests  that  are  based  on  processing  may  provide  an  objective  means  of  selecting 
candidates  for  employment  or  for  advanced  education,  thus  fulfilling  the  spirit  of  affirmative 
action  and  equal  opportunity  programs.”  The  present  experiments  show  that  such  a  hope  may  be 
a  reality. 
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